There are two generally accepted methods for a computer application to download a file from a remote location. In the first method, a client computer connects to a server either directly or via a network gateway. The client computer transmits a request for the server to transmit a whole file at once. The server responds to the request by sending the whole file, typically in packet form, to the client computer. In the second method, the client computer transmits a sequence of requests to the server, or a peer computer (such as in a pear to pear network), where each request asks for specific portions of the file.
Requesting specific portions of the file is preferable to requesting the entire file at once because it allows for resuming of the download if the connection is broken. Requesting specific portions of the file also allows for more efficient utilization of available bandwidth because the client computer can request more portions when more bandwidth is available and fewer portions when less bandwidth is available.
The downloaded portions of the same file can be of different sizes, can be received out of order or can overlap previously downloaded portions. This brings a challenge to an anti-virus (AV) application disposed in the network gateway (or host computer) when the application performs anti-virus scanning and inspection of routed traffic. The scanning process involves an AV application that contains a virus reference signature or heuristical pattern that is compared against the content being downloaded from the remote location. In many cases, anti-virus application must scan an entire file to ensure that no viruses are embedded in the file.
Usually, it is not possible to download the whole file for anti-virus scanning after a portion of the file is requested. The whole file can be very large, requiring significant network bandwidth and time to complete the download. Existing AV solutions either attempt to download the entire file before scanning or limit the scanning to the content of the downloaded portions. Inspecting only a portion of the content downloaded is not sufficient to detect a virus. The virus signature may be spread over two or more portions of the file and may not be identifiable when each file portion is scanned separately.